Skip to content

hey - #3

Open
icedTet wants to merge 386 commits into
Tet-Dev:pullfrom
imagescript:master
Open

hey#3
icedTet wants to merge 386 commits into
Tet-Dev:pullfrom
imagescript:master

Conversation

@icedTet

@icedTet icedTet commented Aug 4, 2021

Copy link
Copy Markdown
Member

No description provided.

matmen and others added 30 commits October 16, 2026 19:15
Opaque images resized with bilinear or area resizing (down to about a third
of the size, and enlarging) round to 8 bits between the passes, so the second
pass shares its multiplications between two channels too. Halving a
1920x1080 photo: 15 -> 12 ms, doubling it: 80 -> 61 ms. Results stay within
1 of the previous ones (1313 corpus images). `ResizeOptions.fast: false`
keeps the previous resampling.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YJgRK5qpwJEbHPuFv6kgAQ
With `fast` (the default), transparent images are premultiplied to 8 bits,
like browsers do, and both passes multiply red+blue and green+alpha as
16-bit lanes; a reciprocal table unpremultiplies. Halving a 1920x1080 photo
with alpha: 43 -> 24 ms, doubling it: 260 -> 119 ms. Over 178 transparent
corpus images the composited result stays within about 2 steps of
`fast: false` (alpha within 2); colors of almost transparent pixels keep
less precision.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YJgRK5qpwJEbHPuFv6kgAQ
- resampleOpaqueFast and resampleTransparentFast become one resampleFast
  (red+blue and green+alpha lanes work for opaque pixels too), split into
  a vertical and a horizontal pass. Transparency in the middle of an image
  is found by the vertical pass, which then reruns premultiplied.
- src/util/alpha.ts: isOpaque, premultiply8 and unpremultiply8 (lookup
  table, rounding like the clamped arrays), used by resample and blur;
  one littleEndian flag.
- Blur results are unchanged (corpus). Fast resizing of transparent images
  differs by 1 only where unpremultiplying hits exactly .5 (0.003-0.007% of
  the values). Doubling a photo with alpha: 117 -> 103 ms.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YJgRK5qpwJEbHPuFv6kgAQ
- predictRun predicts (encoding) or restores (decoding) a run of pixels
  with one loop per mode, instead of predict() with a switch per pixel.
  The encoder's mode search and the decoder's inverse predictor use it.
- residual costs come from a 256-entry table instead of branches.
- tokenize hashes every pixel pair once instead of on search and insert.

Output is identical (VP8L data of 1298 corpus encodes, pixels of 153
WebP decodes). 1920x1080 photo: encoding 1220 -> 735 ms, decoding
95 -> 82 ms.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YJgRK5qpwJEbHPuFv6kgAQ
`RotateOptions.fast` (default true): angles that aren't multiples of 90°
sample 8-bit premultiplied pixels with 8-bit weights, red+blue and
green+alpha sharing a multiplication (src/util/rotate.ts). The source
gets a transparent border of one pixel, so taps outside it need no
checks and edges fade as before; each row only visits the span that hits
the source. unpremultiply8 writes fully transparent pixels directly.

1920x1080 photo by 30°: 119 -> 51 ms (alpha: 114 -> 61 ms). Over the
corpus (4 angles), opaque pixels stay within 1 of `fast: false`, what
shows of translucent ones (also the faded edges) within about 2.5 steps,
alpha within 2. rotate-45 targets regenerated.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YJgRK5qpwJEbHPuFv6kgAQ
- renderText takes a font or an array of fonts by priority: every
  character comes from the first font that has a glyph for it, the first
  font sets the line height. Kerning applies within a font, whole pixel
  placement follows each glyph's own font's hinting.
- src/render/color.ts parses COLR version 0 layers with the first CPAL
  palette, and CBDT/CBLC bitmaps (index formats 1-5, image formats 17-19)
  from the strike closest to the size. Fonts with only bitmaps load.
- Color glyphs render into a premultiplied RGBA layer over the text:
  layers through the rasterizer, PNGs decoded and resampled. The text
  color's alpha applies to them, layers in the text color take its color;
  outlines go around them too.
- Default ignorable code points (joiners, variation selectors, tags) take
  no room, as HarfBuzz hides them.
- Tests with subsets of Noto Color Emoji (OFL) and Twemoji Mozilla
  (CC-BY 4.0), license notices in tests/fonts.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YJgRK5qpwJEbHPuFv6kgAQ
- src/render/gsub.ts applies the default features (ccmp, rlig, liga,
  clig, calt): single, multiple, ligature, contextual and chained
  contextual substitutions (formats 1-3) and extension lookups, with
  lookup flags from GDEF and nested lookups. A memo per lookup skips
  glyphs no subtable starts at.
- Paragraphs are shaped: graphemes (extending characters, joined
  sequences, regional indicator pairs) go to the first font that has all
  of their visible characters, preferring color glyphs before FE0F and
  others before FE0E; each font's run is substituted. Invisible characters
  that weren't substituted show nothing.
- Fallback fonts load when the text first needs them; glyphs, including
  decoded and scaled emoji bitmaps, are kept per font, size and hinting
  across calls (a few sizes per font).
- Tests: skin tones, flags, keycaps and families are single glyphs, ❤️
  picks the emoji font's red heart; emoji targets regenerated.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YJgRK5qpwJEbHPuFv6kgAQ
The stroke covers the glyphs as well (it's a distance field from them), and
was drawn under them with one minus the fill's alpha: with a translucent
text color it showed through. It's now masked by the glyphs' coverage
(text and color glyphs), which is the same for opaque text.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YJgRK5qpwJEbHPuFv6kgAQ
The outline was the distance to the nearest pixel at least half covered,
so its edge followed that pixel staircase (bumpy at larger widths). Near
the outline's edge, the distance now goes to the edge points of the
pixels around the nearest covered one (3x3): from the pixel center along
the coverage gradient to where a straight edge would cover it as much,
like Gustavson's antialiased distance transform. The distance transform
also returns the nearest sample for that.

Against text rendered at 8x and scaled down, the mean error of the
outline's coverage drops from about 0.3 to 0.07-0.1 (Source Sans, Open
Sans, stroke widths 1-24). Distances beyond the outline are left out of
the row pass, so it costs about as much as before (64px text, 3px
outline: 6.6 -> 7.9 ms). effects target regenerated.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YJgRK5qpwJEbHPuFv6kgAQ
src/render/colr.ts renders the paint graph of COLR version 1 base glyphs
(BaseGlyphList, LayerList, ClipList) into premultiplied float RGBA:
- all 32 paint formats, variable ones with their default values
- linear (with the p2 rotation), two point conical radial and sweep
  gradients (biased angles), with pad, repeat and reflect
- PaintGlyph masks through the text rasterizer, after the transform
- all 28 composite modes (Porter-Duff, separable and HSL blends)
- the text color for palette index 0xffff
Layers carry the box they drew in, so compositing stays within it, and
source over has its own loop. Glyphs render at 2x and are averaged down:
layers sharing an edge would otherwise each be antialiased there and the
ones below would show through (dark seams on Noto's emoji).

COLR version 1 glyphs win over bitmaps and version 0 layers. The CPAL
palette reader is shared with version 0.

Tests: a Noto-COLRv1 subset (OFL) in the emoji tests, and a font built
with fontTools (tests/fonts/colrv1-paints.py) with one glyph per paint
feature.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YJgRK5qpwJEbHPuFv6kgAQ
At 2x, the seams where layers share an edge (Noto's flags) still showed;
at 4x they look like the bitmap version. To pay for it:
- layers hold only their box instead of the whole canvas (at 4x a 64px
  emoji's canvas is about 1 MB per layer)
- a glyph over a solid color, the most common paint, is the mask in that
  color without rendering and masking a filled layer
- gradients write from the color table directly, without a closure and a
  subarray per pixel
A new emoji glyph at 40-78px takes about 5.5 ms (10 ms before these, at
4x); output differs only by rounding (3 values of one pixel at alpha 8).
COLRv1 targets regenerated.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YJgRK5qpwJEbHPuFv6kgAQ
- damaged glyf data: non-increasing contour end points leave the glyph
  empty, composites can't contain themselves and have a command budget,
  the hinter likewise (cycles, at most 65536 points)
- COLR v1 masks rasterize within the canvas only (the rasterizer splits
  segments at the bitmap edges instead of clamping their end points), the
  canvas stays within two ems of the em box and lowers its supersampling
  for very large glyphs
- RangeErrors from reading damaged font data become "Font: damaged or
  truncated font data"; text over 2^28 pixels throws; text without ink
  (empty, spaces) renders a transparent pixel wide instead of throwing
- graphemes are composed (NFC) or decomposed (NFD) where that makes a
  font have them, like HarfBuzz
- sbix color bitmaps (PNG, dupe)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YJgRK5qpwJEbHPuFv6kgAQ
- tests/fonts/sbix.py builds sbix.ttf (a PNG with an origin offset, and
  a 'dupe' of it); the bitmap and its placement are checked
- empty text renders a pixel wide, combining sequences like the composed
  characters
- damaged fonts (truncated, bytes changed, and the damaged Noto-COLRv1
  subset that ran out of memory before) render or throw a font error
  within 3 seconds
- changelog entries for sbix, normalization and the robustness fixes

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YJgRK5qpwJEbHPuFv6kgAQ
- DecodeOptions.maxPixels / AnimationDecodeOptions.maxPixels (default
  16383², like sharp): every decoder checks the size it read before
  allocating (PNG, APNG, JPEG, TIFF, WebP/VP8/VP8L, TGA, GIF); animations
  count every frame's canvas, GIFs also the frames' indices while
  parsing. A 250-byte JPEG of 65500² and a 115 kB GIF with 5000 frames on
  4000x4000 killed the process before.
- TIFF: raw samples at most 8 bytes per pixel of the limit, tiles at most
  4 times the image (a damaged SamplesPerPixel took 6 GB).
- JPEG: at most 4 components (each took memory the size of the image).
- RangeErrors from damaged data become "Image: damaged or truncated
  image data" (cause kept); exceeded limits stay RangeErrors (LimitError).
- PNG: rows unfiltered in place, Image.decode keeps the decoded pixels
  (8000x8000: 1272 -> 785 MB), still PNGs in Animation.decode decode like
  images (5.8 s -> 0.57 s), APNG frames that replace pixels copy rows.
- blur/sharpen: infinite radii throw instead of hanging, box windows are
  summed in time independent of the radius (blur(1e9) took minutes).

Found with fuzzing (damaged corpus images, crafted bombs per format, bad
parameters to every public method); decoded pixels of the whole corpus
are unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YJgRK5qpwJEbHPuFv6kgAQ
- elements rendered through <use> references are counted, more than
  100,000 throw (like librsvg's limit): nine levels of ten uses each made
  a billion rectangles from a 1 kB file and killed the process
- dash patterns of more than a million dashes stroke the path solid, as
  Skia does (stroke-dasharray="0.0001" on a long line took any memory)
- the document walk that collects ids and style sheets uses a stack, so
  100,000 nested elements no longer overflow it

Found with crafted SVGs and 540 damaged variants of the test SVGs (no
hangs or slow cases left); test renders are unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YJgRK5qpwJEbHPuFv6kgAQ
…mula

src/util/resample.ts computes its coefficients as Pillow's
precompute_coeffs and uses its box, bilinear and bicubic filters: the
header and THIRD_PARTY_NOTICES.md now carry Pillow's copyright and
MIT-CMU license. The box sizes of the blur follow Peter Kovesi's
formula, as in Ivan Kuckir's boxesForGauss.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YJgRK5qpwJEbHPuFv6kgAQ
Groundwork for text in any script, not used by renderText yet:
- scripts/unicode.js generates src/render/unicode-data.ts from the
  Unicode Character Database 18.0 (general category, combining class,
  bidi class, brackets and mirroring, line break, East Asian width,
  script, grapheme break, Indic conjunct break, Extended_Pictographic,
  Indic syllabic/positional categories, joining type): varint encoded
  ranges in base64, 67 kB (27 kB gzipped), decoded when first used
  (src/render/unicode.ts)
- src/render/bidi.ts: UAX #9, written after the specification (P2-P3,
  X1-X10, W1-W7, N0 brackets, N1-N2, I1-I2, L1-L2); passes all 770,241
  BidiTest.txt and 91,707 BidiCharacterTest.txt cases
- src/render/linebreak.ts: UAX #14 revision 57 (LB1-LB31, LB15a-d, LB20a,
  LB21a-b, LB28a, LB30a-b); passes all 19,346 LineBreakTest.txt cases
- tests/unicode.js runs samples of the three conformance files
- Unicode License v3 in licenses/ and THIRD_PARTY_NOTICES.md

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YJgRK5qpwJEbHPuFv6kgAQ
- the Unicode tables are decoded with Uint8Array.fromBase64 or atob
  instead of a JS base64 decoder
- crc32 uses node:zlib's crc32 (Node, Bun, Deno) from 64 bytes on,
  3-4x faster; shorter buffers stay with slice-by-16
- without node:zlib (browsers), compression levels 1-6 use
  CompressionStream (2.3x faster than the JS deflate); its level is
  fixed, so level 0 and 7-9 stay with the JS deflate

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YJgRK5qpwJEbHPuFv6kgAQ
…rapping

A port of HarfBuzz 14.6's OpenType shaping (Old MIT, credited in the
file headers and THIRD_PARTY_NOTICES.md):
- buffer.ts: the in/out buffer with HarfBuzz's cluster merging
- layout.ts: GDEF, script/language/feature selection with feature
  variations at the default instance, the feature map with stages and
  masks, every GSUB (1-8) and GPOS (1-9) lookup type with HarfBuzz's
  matching (ZWJ/ZWNJ, per-syllable, ligature components, mark
  attachment to ligature components, cursive chains)
- shape.ts: unicode properties, dotted circles, native direction,
  mirroring, normalization (decompose/reorder/recompose by what the
  font has), fractions, space fallback, kern table, fallback mark
  positioning, default ignorables
- shapers: Arabic (with presentation-form fallback and stch), Hebrew,
  Thai/Lao, Hangul, Indic, Khmer, Myanmar and the Universal Shaping
  Engine, their syllable grammars as longest-match NFA scanners
- scripts/unicode.js also generates the Indic/USE categories (after
  HarfBuzz's generators, with Microsoft's USE data in scripts/ms-use),
  canonical decompositions, Arabic presentation forms, vowel
  constraints; property values are translated to short names

Identical to HarfBuzz (glyphs, clusters, positions) on the UDHR in 29
scripts, ~300,000 random sequences over 30 Noto fonts, 15,000 emoji
sequences in the color test fonts and 12,000 with the Latin test fonts.

renderText now analyzes each paragraph (UAX #9 levels, UAX #29
graphemes for font fallback, script itemization with paired brackets,
UAX #14 break opportunities), shapes runs of one level, script and font
with surrounding context, wraps at break opportunities (graphemes for
char wrapping and overlong words), and shapes and reorders every line
by itself (L1/L2). Hinted glyphs keep their hinted advances with the
positioning's adjustments rounded cumulatively. The old GSUB and GPOS
kerning code is removed.

Tests: tests/shaping.js compares 360 texts with HarfBuzz's output for
subsets of Noto fonts (scripts/shaping-reference.py), tests/unicode.js
adds GraphemeBreakTest, tests/font.js a bidirectional render test.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YJgRK5qpwJEbHPuFv6kgAQ
Saves about 5 ms when importing the library.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YJgRK5qpwJEbHPuFv6kgAQ
Colors, coordinates, sizes, angles, factors, encoder levels, the font
size and stroke, frame durations/offsets and loop counts go through
util/options.ts' finite(), which converts like before (numeric strings
work) and names the argument. NaN colors drew black, NaN coordinates
the first pixel, infinite factors saturated everything. TextLayout
keeps Infinity as "no limit", NaN throws instead of meaning no limit.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YJgRK5qpwJEbHPuFv6kgAQ
Windows checkouts turned them into CRLF, so the last break marker of
each test line had a CR and failed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YJgRK5qpwJEbHPuFv6kgAQ
Duplication:
- font.ts: one parser for a simple glyph's points (used by the outlines
  and the hinter) and one for composite components, one loca lookup
  instead of three
- util/bytes.ts: the growing byte buffer of the GIF, JPEG and WebP
  (VP8L) encoders; their outputs are byte for byte the same
- shapers: lazyScanner() instead of the same lazy wrapper four times

Performance (renderText of ~1500-2000 characters wrapped at 800 px):
- the syllable scanners run as DFAs built while scanning instead of
  simulating the NFA, the lookups' coverage memo is a typed array,
  ligature sets are parsed once, the NFD form of a character is only
  computed when its font lacks it, text is colored a word per pixel:
  Latin 29.8 -> 23.2 ms, Arabic 27.5 -> 19.2 ms, Devanagari 41.6 ->
  23.0 ms
- glyph bitmaps are clipped once per glyph instead of per pixel

Also: unused parameters and a re-export removed, documentation of
render() and Image.renderText updated for the shaping.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YJgRK5qpwJEbHPuFv6kgAQ
- resample.ts: the premultiplied pixels of the vertical retry were
  assigned to a variable nothing reads afterwards
- hinting.ts: the parts and placed arrays of composite glyphs were only
  written
- zlib.ts: every block is inflated to its end before the next header,
  so the "no table yet" check (from fflate's streaming inflate) was
  always true; the tables are local to a block now. Checked against
  node:zlib on 400 streams (stored, fixed and dynamic blocks, levels
  0-9)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YJgRK5qpwJEbHPuFv6kgAQ
ESLint (typescript-eslint strict type-checked) with SonarJS, knip and a
regex timing harness over every regular expression of src:

- svg.ts: regular expressions that backtracked on hostile input are
  replaced: the stylesheet (blocks and comments) is scanned with
  indexOf, url() references by a small scanner, numbers use a pattern
  whose alternatives can't match the same digits twice, colors and
  lengths are trimmed instead of matching spaces twice. A 200,000
  character style without braces took 60 s, url(#a followed by spaces
  was exponential; all of them take a few ms now (regression tests).
  The 1952 sample SVGs render the same; an unclosed style rule at the
  end applies now, like in browsers
- types: the TIFF tags, quantizer lookup and SVG arrays are typed
  (no any), Shaper hooks are function properties (no unbound methods),
  redundant assertions removed
- layout.ts: one coverage list helper for contexts and reverse
  chaining; text.ts, buffer.ts, machine.ts: unused helpers removed
- one statement per line where single-line blocks read like bugs
  (vp8.ts, hinting.ts), substr -> slice, prefer-const

The remaining findings are style (cognitive complexity, nested
ternaries, const enum comparisons) or defensive checks TypeScript
considers impossible (index lookups, input from JavaScript callers).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YJgRK5qpwJEbHPuFv6kgAQ
deno doc (and so JSR) ignores @internal, which only tsc's stripInternal
reads, and shows the __internal__ members of Image, Frame and Animation.
Their doc comments also get @ignore, on its own line (deno doc reads
"@internal @ignore" on one line as @internal with the text "@ignore").
deno doc --lint is clean now: toString(), the encodePNG overloads and
the TextLayout constructor got doc comments.

tests/rotate.js: a missing semicolon made Bun parse the IIFE as a call
of the panic function's result.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YJgRK5qpwJEbHPuFv6kgAQ
tests/run.js runs every test with the runtime running it, so
`deno run -A tests/run.js` and `bun tests/run.js` run the suite as
`npm test` does in Node; the CI tests both on Linux, macOS and Windows.
tests/package.json declares the tests CommonJS, which Deno doesn't
assume for .js files, and the hashes take a Uint8Array copy of the
bitmaps (Deno's node:crypto rejects Uint8ClampedArray).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YJgRK5qpwJEbHPuFv6kgAQ
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YJgRK5qpwJEbHPuFv6kgAQ
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants